Virtual reality surgical device

ABSTRACT

A system for use in surgery includes a central body, a visualization system operably connected to the central body, a video rendering system, a head-mounted display for displaying images from the video rendering system, a sensor system, and a robotic device operably connected to the central body. The visualization system includes at least one camera and a pan system and/or a tilt system. The sensor system tracks the position and/or orientation in space of the head-mounted display relative to a reference point. The pan system and/or the tilt system are configured to adjust the field of view of the camera in response to information from the sensor system about changes in at least one of position and orientation in space of the head-mounted display relative to the reference point.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application is a national stage of International Application No.PCT/US2015/29247, filed May 5, 2015, which claims the benefit of U.S.Provisional Patent Application No. 61/988,498, entitled “Method forNatural Human-Like Motion and Human Interface in Surgical Robotics”filed on May 5, 2014, and Provisional Patent Application No. 62/136,883,entitled “Virtual Reality Surgical Device”, filed on Mar. 23, 2015, allof which are hereby incorporated by reference in their entirety.

BACKGROUND

Field of Invention

This application generally relates to minimally invasive surgery and tovirtual reality systems.

Description of Related Art

Since its inception in the early 1990s, the field of minimally invasivesurgery has grown rapidly. While minimally invasive surgery vastlyimproves patient outcome, this improvement comes at a cost to thesurgeon's ability to operate with precision and ease. Duringlaparoscopy, the surgeon must insert laparoscopic instruments through asmall incision in the patient's abdominal wall. The nature of toolinsertion through the abdominal wall constrains the motion oflaparoscopic instruments as laparoscopic instruments cannot moveside-to-side without injury to the abdominal wall. Standard laparoscopicinstruments are limited to four axes of motion. These four axes ofmotion are movement of the instrument in and out of the trocar (axis 1),rotation of the instrument within the trocar (axis 2), and angularmovement of the trocar in two planes while maintaining the pivot pointof the trocar's entry into the abdominal cavity (axes 3 and 4). For overtwo decades, the majority of minimally invasive surgery has beenperformed with only these four degrees of motion.

Existing robotic surgical devices attempted to solve many of theseproblems. Some existing robotic surgical devices replicate non-roboticlaparoscopic surgery with additional degrees of freedom at the end ofthe instrument. However, even with many costly changes to the surgicalprocedure, existing robotic surgical devices have failed to provideimproved patient outcome in the majority of procedures for which theyare used. Additionally, existing robotic devices create increasedseparation between the surgeon and surgical end-effectors. Thisincreased separation causes injuries resulting from the surgeon'smisunderstanding of the motion and the force applied by the roboticdevice. Because the degrees of freedom of many existing robotic devicesare unfamiliar to a human operator, surgeons must train extensively onrobotic simulators before operating on a patient in order to minimizethe likelihood of causing inadvertent injury.

To control existing robotic devices, a surgeon sits at a console andcontrols manipulators with his or her hands and feet. Additionally,robot cameras remain in a semi-fixed location, and are moved by acombined foot and hand motion from the surgeon. These semi-fixed cameraswith limited fields of view result in difficulty visualizing theoperating field.

Other robotic devices have two robotic manipulators inserted through asingle incision. These devices reduce the number of incisions requiredto a single incision, often in the umbilicus. However, existingsingle-incision robotic devices have significant shortcomings stemmingfrom their actuator design. Existing single-incision robotic devicesinclude servomotors, encoders, gearboxes, and all other actuationdevices within the in vivo robot. This decision to include the motorsand gearboxes within the patient's body has resulted in large robotswith limited capability. Such a large robot must be inserted through alarge incision, thus increasing risk of herniation, risk of infection,pain, and general morbidity. The incision size required for someexisting devices is between 1.5 and 2 inches—an incision size similar toopen surgery. Additionally, it is unlikely that the size of thesedevices will ever significantly decrease due to the inclusion of motors,gears, etc. within the in vivo devices. This increased incision sizeresults in significantly increased injury to the patient and vastlyreduces the practicality of existing devices.

Existing single incision devices also have limited degrees of freedom.Some of these degrees of freedom are non-intuitive to a human, forexample elongation of the arm during a procedure. These degrees offreedom require a user interface where the surgeon must makenon-intuitive learned movements similar the movements existingmulti-incision devices.

Human-Like Robotics

A few people have previously suggested the idea of surgical roboticsdesigned to replicate the degrees of freedom of a human arm. However,all existing designs include extraordinarily complex gearboxes and geartrains all placed within the robotic arms. As a result of thesegearboxes and gear trains, existing human-like arms are both difficultto manufacture, large in size, and low in speed. In addition, noprevious inventors of human-like robotics describe human-machineinterfaces designed to fully utilize the advantages of human-likerobotics. Without a proper human-machine interface, a human like armprovides little or no advantage over alternative robotic designs.

BRIEF SUMMARY OF THE INVENTION

In one embodiment the invention includes a system for use in surgerycomprising a central body, a visualization system operably connected tothe central body comprising, at least one camera, at least one of a pansystem and a tilt system, a video rendering system for generating imagesbased on information from the at least one camera, a head-mounteddisplay which displays images received from the camera, a sensor systemto track at least one of the position in space of the head-mounteddisplay relative to a reference point and the orientation in space ofthe head-mounted display relative to the reference point, wherein atleast one of the pan system and the tilt system are configured to adjustthe field of view of the camera in response to information from thesensor system about changes in at least one of position and orientationin space of the head-mounted display relative to the reference pointand, a robotic device operably connected to the central body. In thesurgery system, the video rendering system further configured fordigitally adjusting the field of view of the generated images based oninformation from the sensor system. The surgery system may also comprisea second camera. In the surgery system comprising two cameras the imagesgenerated by the video rendering system may also comprise stereoscopicimages based on information from the first and second cameras.

The surgery system may also comprise a plurality of cameras. In thesurgery system comprising a plurality of cameras the video renderingsystem may also generate the images based on software interlacing ofsignal information from the plurality of cameras.

The surgery system may also comprise at least one sensor to measure atleast one of the position and orientation of the camera. The surgerysystem comprising two cameras, in an insertion configuration,cross-sectional dimensions of the visualization system in a plane normalto an insertion axis are smaller than a center distance between thefirst camera and the second camera along the insertion axis.

The robotic device of the surgery system may further comprise a firstrotational actuator for rotating one portion of the robotic device withrespect to another portion of the robotic device, and a first hingedactuator for changing the angle between one portion of the roboticdevice and another portion of the robotic device operably coupled to thefirst rotational actuator. The robotic device may further comprise apositional actuator for changing the position of the robotic devicerelative to the central body such that the robotic device may be used oneither a first side or a second side of the central body.

In other embodiments, the robotic device of the surgery system mayfurther comprise a first rotational actuator for rotating one portion ofthe robotic device with respect to another portion of the roboticdevice, a first hinged actuator for changing the angle between oneportion of the robotic device and another portion of the robotic deviceoperably coupled to the first rotational actuator, a second rotationalactuator for rotating one portion of the robotic device with respect toanother portion of the robotic device operably coupled to the firsthinged actuator, a second hinged actuator for changing the angle betweenone portion of the robotic device and another portion of the roboticdevice operably coupled to the second rotational actuator, a thirdrotational actuator for rotating one portion of the robotic device withrespect to another portion of the robotic device operably coupled to thesecond hinged actuator, a third hinged actuator for changing the anglebetween one portion of the robotic device and another portion of therobotic device operably coupled to the third rotational actuator, and asurgical end-effector operably coupled to the third hinged actuator.

In still other embodiments, the surgery system may further comprise asecond robotic device comprising a fourth rotational actuator forrotating one portion of the robotic device with respect to anotherportion of the robotic device, a fourth hinged actuator for changing theangle between one portion of the robotic device and another portion ofthe robotic device operably coupled to the fourth rotational actuator, afifth rotational actuator for rotating one portion of the robotic devicewith respect to another portion of the robotic device operably coupledto the fourth hinged actuator, a fifth hinged actuator for changing theangle between one portion of the robotic device and another portion ofthe robotic device operably coupled to the fifth rotational actuator, asixth rotational actuator for rotating one portion of the robotic devicewith respect to another portion of the robotic device operably coupledto the fifth hinged actuator for changing the angle between one portionof the robotic device and another portion of the robotic device, a sixthhinged actuator for changing the angle between one portion of therobotic device and another portion of the robotic device operablycoupled to the sixth rotational actuator, and a surgical end-effectoroperably coupled to the sixth hinged actuator.

In another aspect the invention includes a robotic surgical devicecomprising a first cable driven rotational actuator for rotating oneportion of the robotic device with respect to another portion of therobotic device, a first cable driven hinged actuator for changing theangle between one portion of the robotic device and another portion ofthe robotic device operably coupled to the first cable driven rotationalactuator, a second cable driven rotational actuator for rotating oneportion of the robotic device with respect to another portion of therobotic device operably coupled to the first cable driven hingedactuator, a second cable driven hinged actuator for changing the anglebetween one portion of the robotic device and another portion of therobotic device operably coupled to the second cable driven rotationalactuator, a third cable driven rotational actuator for rotating oneportion of the robotic device with respect to another portion of therobotic device operably coupled to the second cable driven hingedactuator for changing the angle between one portion of the roboticdevice and another portion of the robotic device, a third cable drivenhinged actuator for changing the angle between one portion of therobotic device and another cable driven portion of the robotic deviceoperably coupled to the third rotational actuator, and a surgicalend-effector operably coupled to the third cable driven hinged actuator.In some embodiments the robotic device is placed partially within apatient's body. In other embodiments, the robotic device is placed fullywithin a patient's body.

In still another aspect, the invention includes a robotic actuatorcomprising, a first body comprising a proximal connection componentcoupling the robotic actuator to proximal systems and a first bearingsurface, a second body comprising a distal connection component couplingthe robotic actuator to distal systems and a second bearing surfaceforming a bearing with the first bearing surface whereby the bearingconstrains the motion of the first body relative to the motion of thesecond body in at least one degree of freedom, a pulley or capstanoperably coupled with the first body or the second body, an actuatorcable configured to actuate the pulley or capstan, and at least onecontoured surface defined by the robotic actuator and forming acontoured pathway to allow a plurality of additional cables to passthrough the pathway from systems coupled to the proximal connectioncomponent to systems coupled to the distal connection component whereina shape and a position of the pathway is such that lengths of theadditional cables remain substantially constant for substantially anentire range of motion for which the robotic actuator is used.

The invention also includes a robotic actuator comprising, a first bodycomprising a proximal connection component, a second body comprising adistal connection component, a bearing system constraining the motion ofthe first body relative to the second body in all degrees of freedomexcept rotation about one axis perpendicular to the distal-proximal axisof the robotic actuator, a pulley or capstan operably coupled with thefirst body or the second body, an actuator cable configured to actuatethe pulley or capstan, and at least one contoured surface defined by therobotic actuator and forming a contoured pathway to allow a plurality ofadditional cables to pass through the pathway from systems coupled tothe proximal connection component to systems coupled to the distalconnection component wherein a shape and a position of the pathway issuch that lengths of the additional cables remain substantially constantfor substantially an entire range of motion for which the roboticactuator is used.

The invention also includes a robotic actuator comprising, a first bodycomprising a proximal connection component a second body comprising adistal connection component, a bearing system constraining the motion ofthe first body relative to the second body in all degrees of freedomexcept rotation about the distal-proximal axis of the robotic actuator,a pulley or capstan operably coupled with the first body or the secondbody, an actuator cable configured to actuate the pulley or capstan, anda hole defined by the robotic actuator with an inner diameter of atleast three times the diameter of the actuator cable configured suchthat additional cables may pass through the hole from systems coupled tothe proximal connection component to systems coupled to the distalconnection component wherein a shape and a position of the hole is suchthat lengths of the additional cables remain substantially constant forsubstantially an entire range of motion for which the robotic actuatoris used.

A further aspect of the invention includes a surgical graspercomprising, a main grasper body, a first grasper jaw operably coupled tothe grasper body, a second grasper jaw operably coupled to the grasperbody, an actuation cable, and a linkage mechanism coupling at least oneof the first grasper jaw and the second grasper jaw with the actuationcable wherein the linkage provides for non-linear movement of the distalend of the first grasper jaw or the second grasper jaw in response tomovement of the actuation cable. The surgical grasper may furthercomprise a strain gauge fixed to at least one of the main grasper body,the first grasper jaw, the second grasper jaw, the actuation cable, andthe linkage mechanism whereby the strain measured by the strain gaugemay be used to calculate the force between the distal end of the firstgrasper jaw and the distal end of the second grasper jaw. In anotherembodiment, the surgical grasper may further comprise at least one of aspring operably coupled with at least one of the first grasper jaw andthe second grasper jaw. In another embodiment, the surgical grasper mayfurther comprise at least one of software and hardware control loops forcontrolling at least one of the force of the grasper jaws and theposition of the grasper jaws. In another embodiment, the surgicalgrasper may further comprise at least one of a servomotor operablycoupled with the actuation cable. In another embodiment, the surgicalgrasper may further comprise at least one of a position sensor wherebythe position sensor measures the position of at least one of the firstgrasper jaw and the second grasper jaw.

The surgical grasper may further comprise a device to provide hapticfeedback whereby the calculated force is used to determine hapticfeedback force.

BRIEF DESCRIPTION OF FIGURES

Note that numbed items remain consistent across all figures. Itemsnumbered with the same number are either the same item, or identicalcopies of the item. Items numbered with different numbers are eitherparts of different design, or are occasionally identical parts servingdifferent purposes.

FIG. 1A is a front isometric view of one embodiment of a two-arm roboticsurgical device as configured for use.

FIG. 1B is a rear isometric view of one embodiment of a two-arm roboticsurgical device as configured for use.

FIG. 2A is an exploded isometric view of the right arm of a roboticsurgical device in one embodiment.

FIG. 2B is a diagram of one embodiment of a two-arm robotic surgicaldevice with first actuators oriented in a first position.

FIG. 2C is a diagram of one embodiment of a two-arm robotic surgicaldevice with first actuators oriented in a second position.

FIG. 3A is an isometric view of one embodiment of a two-arm roboticsurgical device as configured for insertion through trocar.

FIG. 3B is an isometric view of a trocar with a two-arm robotic surgicaldevice fully inserted according to one embodiment.

FIG. 3C is a diagram of a trocar with inner sleeve in one embodiment.

FIG. 3D is a diagram of one embodiment of a trocar showing patent lumen.

FIG. 4 is an isometric view of alternate embodiment showing insertionbody.

FIG. 5A is an isometric view of trocar sleeve with camera according toone embodiment.

FIG. 5B is an isometric view of trocar sleeve with camera duringinsertion through trocar according to one embodiment.

FIG. 5C is an isometric view of trocar sleeve with camera in placeduring use according to one embodiment.

FIG. 6A is a front view of camera as configured for use according to oneembodiment.

FIG. 6B is a front view of camera as configured for insertion intoabdomen according to one embodiment.

FIG. 7A is an isometric view of center connection component according toone embodiment.

FIG. 7B is an exploded view of center connection front view according toone embodiment.

FIG. 7C is an exploded view of center connection rear view according toone embodiment.

FIG. 8A is an isometric view of hinge actuator according to oneembodiment.

FIG. 8B is an exploded view of hinge actuator according to oneembodiment.

FIG. 9A is diagram of one embodiment of a section of hinge actuatoractuated to 0 degrees.

FIG. 9B is diagram of one embodiment of a section of hinge actuator inactuated to 30 degrees.

FIG. 10A is a side view of rotary actuator first side according to oneembodiment.

FIG. 10B is a side view of rotary actuator second side according to oneembodiment.

FIG. 10C is an exploded view of rotary actuator according to oneembodiment.

FIG. 10D is a section view of rotary actuator according to oneembodiment.

FIG. 11A is an isometric view of grasper according to one embodiment.

FIG. 11B is an exploded view of grasper according to one embodiment.

FIG. 11C is a section of grasper while closed, according to oneembodiment.

FIG. 11D is a section of grasper while open, according to oneembodiment.

FIG. 12A is a diagram showing opening of grasper according to oneembodiment.

FIG. 12B is a diagram showing closing of grasper according to oneembodiment.

FIG. 13 is an isometric view of an extension segment according to oneembodiment.

FIG. 14A is an isometric view of four-arm robotic surgical systemaccording to one embodiment.

FIG. 14B is an isometric view of four-arm robotic surgical trocaraccording to one embodiment.

FIG. 15 is a diagram showing placement of MEMS sensors on user with userwearing a virtual reality headset according to one embodiment.

FIG. 16 is a block diagram of one embodiment of the virtual realityrobotic system.

FIG. 17 is a front view of a device with separate robotics and camerasystem according to one embodiment.

DETAILED DESCRIPTION

While the present system is designed for use by a surgeon within theabdominal cavity, many alternative uses of the device are possible. Forexample, a user might be a physician assistant, nurse, surgical aid, orany other surgical personnel. Additionally, the device could be disposedwithin any part of a patient's body, and future embodiments could bedesigned to be much smaller so as to allow for use within smaller areasof a patient's body. Both smaller and larger devices could be fabricatedto be used in areas such as the paranasal sinuses, colon, stomach, orany other area within the human body. Micro-fabrication using MEMS orother means could allow for a device to be positionable within extremelysmall areas such as human blood vessels. Alternatively, the system couldbe used to gain excellent dexterity and visualization even during openprocedures with the device positioned partially or entirely outside ofthe human body.

In other embodiments, the device is used for non-surgical or non-medicaltasks such as micro-fabrication, assembly of parts, bomb defusing, orany other task requiring fine motor skills. Alternative embodiments ofthe device could be fabricated with arms that are human-sized or evenlarger-than-life allowing humans to perform tasks for which they are toosmall, too weak, or otherwise unable. Obviously, in such embodiments,the user may not necessarily be a surgeon.

The following define words as used in the detailed description andclaims:

Surgeon: a user of the device

Abdominal cavity: any enclosed or semi-enclosed area into which thedevice is inserted

Abdominal wall: wall partially or fully enclosing aforementionedabdominal cavity

Trocar: tube for insertion of device through aforementioned abdominalwall

Distal: closer to the end-effector of a robotic arm

Proximal: further from the end-effector of a robotic arm

Overall Device Design

FIG. 1A shows an isometric view of one embodiment of our device asdisposed within the patient's abdominal cavity. This device comprises aconduit 100 connected to a central body 101. The central body isdisposed within the abdomen of a patient. The conduit is comprised of ahollow tube traversing the abdominal wall, thus bringing power, signal,control cables, irrigation, vacuum, and any other systems from systemsoutside the patient's body to inside the patient's body. In someembodiments, conduit 100 includes multiple lumens to separate varioussystems and cables, and to provide independent fluid channels. In otherembodiments, the conduit comprises multiple interlocking segments suchthat the conduit is flexible while all control cables are slack, yetbecomes rigid when tension is applied to control cables within theconduit. This design would function in a similar manner to a campingtent support pole with interlocking segments joined by a cable. When thecable is tensioned and segments are moved together, they form a rigidpole. In yet another embodiment, the conduit is a rigid tube bent into anon-linear shape.

FIG. 1A further shows a right robotic arm 103 and a left robotic arm 104attached to the central body 101. Each of these robotic arms comprisesmultiple actuators connected to form a human-like robotic arm. Theactuators of each robotic arm are assembled to form a robotic shoulder105, a robotic elbow 106, a robotic wrist 107, and a surgicalend-effector 109.

FIG. 1B shows a camera assembly 102 attached to the central body 101.This camera body is positioned such that it is located approximatelycentrally between the robotic shoulders 105 and slightly above therobotic shoulders. The camera body is positioned such that the ratiobelow remains true.

$\frac{{horizontal}\mspace{14mu}{distance}\mspace{14mu}{between}\mspace{14mu}{human}\mspace{14mu}{shoulders}}{{vertical}\mspace{14mu}{distance}\mspace{14mu}{between}\mspace{14mu}{human}\mspace{14mu}{shoulders}\mspace{14mu}{and}\mspace{14mu}{human}\mspace{14mu}{eyes}}$${approximately}\mspace{14mu}{equals}\frac{{horizontal}\mspace{14mu}{distance}\mspace{14mu}{between}\mspace{14mu}{robotic}\mspace{14mu}{shoulders}}{\begin{matrix}{{vertical}\mspace{14mu}{distance}\mspace{14mu}{between}\mspace{14mu}{robotic}\mspace{14mu}{shoulders}} \\{{and}\mspace{14mu}{camera}\mspace{14mu}{assembly}}\end{matrix}}$

While this ratio for typical humans is roughly equal to 2, it isunderstood that the ratio may vary from person to person. In oneembodiment, the device may be fabricated to exactly match the ratio ofthe surgeon, while in another embodiment a general ratio is maintainedto approximately the proportions of the typical surgeon. In anotherembodiment, the ratio is adjustable either during use or beforeinsertion into the patient. Alternatively, the ratio may beintentionally increased so as to reduce the overall vertical height ofthe device during use. This reduction serves to increase the workingarea within the patient's abdominal cavity. For maximum versatility ofinitial devices, one embodiment is designed with a ratio ofapproximately 4, compromising some human-like feel for increased deviceversatility. It is hypothesized that devices with a ratio between 1 and4 will retain sufficiently human-like view for the surgeon.

With the above ratio, the camera assembly is placed in a position togive a natural, human-like view of the robotic arms. In an alternativeembodiment of the device with only one human-like robotic arm, thecamera is still placed such that it maintains a human-like perspectiveover the arm. Additionally in another alternative embodiment, the camerais moved forward such that the plane formed by the camera assembly 102and the two robotic shoulders 105 is perpendicular to the plane formedby the central body 101 and the two robotic shoulders. Alternatively,camera zoom can give the user the impression of a camera that has beenplaced more forward, or actuators could give the camera body the abilityto move forward during a procedure.

In other embodiments, any of the right robotic arm 103, the left roboticarm 104, and the camera assembly 102 are not attached to the centralbody 101. In these embodiments, individual components of the system areinserted separately into the patient's abdominal cavity. Thesecomponents may be inserted through a single trocar, or through manytrocars. The components may assemble within the abdominal cavity.Alternatively, the components may remain separate, yet positioned suchthat the human-like robotics work in unison with the natural, human-likevisualization. One such embodiment includes a camera system insertedseparately and supported by its own conduit 234 as shown in FIG. 17.

Robotic Arm Design

FIG. 2A shows an isometric view of the right robotic arm 103 (FIG. 1A).This arm is comprised of multiple robotic joints. A first actuator 110(FIG. 2A) is connected to the central body 101 (FIG. 1A). In oneembodiment, the first actuator serves to allow the surgeon to operate oneither side of the device, and to straighten the device for insertionand removal from the abdominal cavity. FIG. 2B shows both the left andright arms' first actuators oriented such that the robotic arms arepositioned to operate on one side of the device. Similarly, FIG. 2Cshows both the left and right arm first actuators oriented such that thesurgical arms are positioned to operate on the second side of thedevice. When operating on the second side of the device, the right armbecomes the left arm, and the left arm becomes the right arm. Thischange is made in the software controlling the device.

Additionally, the camera assembly 102 (FIG. 1A) is able to swivel morethan 180 degrees. This range of motion allows the surgeon to place thedevice anywhere in the patient's abdominal cavity and view the abdomenfrom any orientation. For example, to operate on a patient's gallbladder, a surgeon might place the device on the patient's left andorient the arms and camera facing to the patient's right. Alternatively,for an operation on the stomach, a surgeon might place the device on thepatient's right orienting the arms and camera facing to the patient'sleft. This versatility allows one device to be used for many differentprocedures.

FIG. 2A additionally shows a second actuator 111 connected to the firstactuator 110 and a third actuator 112 connected to the second actuator.The second actuator provides rotary actuation about the axis along thecenter of the arm. The third actuator provides hinged actuation withrotation about an axis perpendicular to the axis of the second actuator.Together, the second and third actuators provide the robotic shoulder105 (FIG. 1A) with two degrees of freedom mimicking those of the humanshoulder's ball joint. These degrees of motion mimic human shoulderabduction/adduction and human arm flexion/extension. In alternativeembodiments, other actuator types may allow for shoulder degrees offreedom including ball joint actuators.

FIG. 2A additionally shows a fourth actuator 113 and a fifth actuator114. The fourth actuator connects the third and fifth actuators andprovides rotary actuation about the axis along the center of the arm.The rotatory actuation of the fourth actuator mimics the motion of humanarm outward/inward rotation. The fifth actuator connects to the fourthactuator and forms the robotic elbow 106 (FIG. 1A). The fifth actuatorprovides hinged actuation with rotation about an axis perpendicular tothe axis of the fourth actuator. The hinged actuator of the fifthactuator mimics the motion of human elbow flexion/extension.

FIG. 2A additionally shows a sixth actuator 115 and a seventh actuator116. The sixth actuator connects the fifth and seventh actuators andprovides rotary actuation about the axis along the center of the arm.The rotatory actuation of the sixth actuator mimics the motion of humanpalm supination/pronation. The seventh actuator connects to the sixthactuator and forms the robotic wrist. The seventh actuator provideshinged actuation with rotation about an axis perpendicular to the axisof the sixth actuator. The hinged actuator of the seventh actuatormimics the motion of wrist extension and flexion. A surgicalend-effector 109 is connected to the seventh actuator. In the oneembodiment, the surgical end-effector provides the surgeon with arobotic grasper with motion similar to pinching of the thumb andforefinger (first and second digits).

With the combination of actuators as described above, the robotic armhas degrees of freedom mimicking that of a human arm. Thus, the arm isable to replicate human arm motions almost exactly. The specific designof both the rotary and elbow actuators as described below enable thismany degree of freedom robotic arm to both mimic human motion and fitthrough a standard 12 mm trocar.

In one embodiment, the rotary actuator does not provide for continuousrotation without limit. Thus the arm cannot perfectly mimic all motionsof the human shoulder without limit. Certain motions, when repeatedmultiple times, would result in the second actuator reaching itshard-limit. To overcome this limitation, computer control algorithmslimit motion of the shoulder joint in one embodiment such thatcontinuous rotation is not required. Software prevents the surgeon frommoving the robotic elbow past an imaginary plane. Continuous rotation isnever required as long as the imaginary plane is placed for each roboticarm such that the axis of each second actuator is coincident with itsrespective arm's plane. This plane may be oriented differently dependingon the needs of each surgery. For example, when operating entirely belowthe device, the planes for both arms may be parallel to the ground suchthat the robotic elbows may never pass above the height of the roboticshoulders. Alternatively, for a surgery out in front of the device, theplanes may be placed perpendicular to the ground.

In one embodiment, the robotic device does not include a degree offreedom mimicking radial/ulnar deviation. While a human arm does havethis degree of freedom our experimentation has found that the degree offreedom is not critical to device function. However, an alternativeembodiment of the device provides for this degree of freedom.

Insertion and Removal of Device

FIG. 3A shows one embodiment of a two-arm robotic surgical device asconfigured for insertion and removal into the patient's abdominalcavity. For insertion, all hinge joints are positioned in a straightorientation as shown in FIG. 3A. In one embodiment, hinge joints arestraightened by removing force on cables, thus allowing all hinge jointsto become slack. Slacked joints are manually straightened as needed. Inan alternative embodiment, the actuators are driven to the straightposition. In some embodiments the actuators may continue to be powered,yet controlled with a damping algorithm simulating free moving actuatorswith damping. In another embodiment joints are actuated into anon-linear orientation for passing through a curved trocar as discussedbelow.

In some embodiments the robotic surgical device can be inserted into theabdomen through a trocar 117. In one embodiment, the trocar is designedwith a cross-sectional profile similar to that of the device duringinsertion. During insertion, the device passes through the trocar withminimal clearance to allow for the smallest possible incision in thepatient's abdominal wall. In alternative embodiments, the device couldbe inserted through standard commercial trocars.

FIG. 3B shows the trocar with the device already inserted. FIG. 3C andFIG. 3D show the trocar and trocar inner sleeve 118. Once inserted, theconduit 100 (FIG. 3B) consumes a portion 119 (FIG. 3D) of thenon-circular trocar 117. In one embodiment, a trocar inner sleeve 118 isplaced to hold the conduit in position, thus leaving a 12 mm circularopening 120 in the trocar. In some embodiments, this inner sleeve formsa gas seal against the trocar. In other embodiments, the inner sleevecontains a rubber check-valve to help maintain a gas seal for surgicalinsufflation. In one alternate embodiment, the trocar is flexible and/orcurved. This flexibility would allow for the passing of a curved deviceor conduit through the trocar.

In some embodiments, the inner sleeve contains a tube connecting a gasport outside the patient and the inside of the patient's abdomen. Thistube and gas port allow for insufflation of the abdominal cavity.Alternatively, the trocar 117 may contain such a tube and gas port toallow for insufflation. In some embodiments, the trocar contains acheck-valve to maintain insufflation pressure prior to insertion of therobotic device. In other embodiments a removable plug blocks thetrocar's opening for insufflation with the robotic device removed.

FIG. 4 shows one embodiment of the device as positioned within aninsertion body 121. This insertion body allows for easy movement of thedevice into the patient's abdomen. The device can be shipped within theinsertion body to prevent the need for the surgeon to manuallystraighten the device's hinge joints. The surgeon simply slides thedistal end of the insertion body 122 through the trocar. The distal endof the insertion body is fashioned to be soft and rounded so as to avoiddamage to tissue during insertion. A surgeon slides the insertion bodyinto the patient's abdominal cavity until he or she meets resistance,indicating contact with the abdominal wall or organs. Upon contact, thesurgeon retracts the insertion body while advancing the device. As thedevice leaves the protection of the insertion body, the flaccid hingejoints bend to allow the device to curl within the abdomen. In someembodiments, the insertion body distal end 122 includes a sensor todetect contact with and/or proximity to the abdominal wall using anystandard means of sensing (capacitance, resistance, conductivity,pressure, etc). In other embodiments the entire insertion body isflexible and/or curved.

An insertion body can also assist in removal of the device. An assistantmay place the insertion body through the trocar. The surgeon can movethe arm closest to the insertion body into the insertion body, and thusthe insertion body slides over the entire device as it is removed fromthe abdomen.

FIG. 5A shows an additional trocar inner sleeve 123. This sleeve isextremely thin, and is inserted into the trocar once the trocar ispositioned traversing the abdominal wall. A camera 124 is attached tothe end of the trocar inner sleeve such that it does not obstruct theopening of the trocar. FIG. 5B shows the additional trocar inner sleeveas it is inserted through the trocar. FIG. 5C shows the additionaltrocar inner sleeve in place and fully inserted in the trocar. Onceinserted, the camera provides visualization of the abdominal cavity andaids in the safe insertion and removal of the device to and from theabdominal cavity. This camera may further comprise a light source aswell as other sensors to assist and acquire data during the procedure.This camera may additionally consist of a plurality of cameras toprovide multiple views within the abdomen.

Camera and Visualization Systems

FIG. 6A shows the camera and visualization system as configured for usewithin the patient's abdominal cavity. FIG. 6B shows the camera andvisualization system as configured for insertion through the trocar. Thecamera system moves between the in-use position shown in FIG. 6A and theinsertion position shown in FIG. 6B by actuation of a hinge joint 126and a ball joint 127. Hinge joint 126 is best visualized in FIG. 7C.Hinge and ball joints can be actuated using any standard means ofactuation, including cables, motors, magnets, electromagnets, etc. Inone embodiment, the hinge joint is actuated by spring with the springactuating the camera into the in-use position shown in FIG. 6A. When thedevice is retracted through the trocar, the direction of force appliedby the end of the trocar forces the spring-actuated hinge joint to moveinto the insertion position shown in FIG. 6B to allow it to fit withinthe trocar or insertion tube.

The camera system shown in FIG. 6A comprises a first camera 128 and asecond camera 129 disposed within, adjacent to, or on top of a camerabody 125. The camera body pivots with two degrees of freedom byactuation of the ball joint 127. This motion in two degrees of freedomforms the camera's pan system and tilt system. The pan system adjuststhe camera's view in the pan axis while the tilt system adjusts thecamera's view in the tilt axis. In alternative embodiments, the balljoint is replaced with two hinge joints or a rotary joint and a hingejoint. In yet another embodiment, a rotary joint rotates the camera bodyabout the vertical axis while tilt motion is provided by digitallyadjusting the camera view. Position sensors accurately measure theposition of each joint that moves the camera body. Position sensors mayinclude any of hall-effect sensors, optical encoders, resistive positionsensors, or any other standard means of measuring position.

In another embodiment, cameras move within the camera body such that oneor both of pan and tilt adjustments are provided by movement of thecameras within the camera body. This adjustment may be used inconjunction with camera body movement, or instead of camera bodymovement. Cameras may move together, or separately. Cameras may move bymotor actuation, cable actuation, or any other standard actuation means.Alternatively, cameras may rotate freely in two degrees of freedom andmove under the direction of a magnetic field created by magnetic coilssurrounding the camera.

In some embodiments, both pan and tilt motion are provided by digitalpan and tilt adjustment. Digital adjustment is provided by cropping thedigital camera image. The cropped image adjusts automatically such thatas a pan or tilt movement is desired, the portion of the image displayedto the user changes, thus creating the illusion of camera movement. Inanother embodiment, a combination of digital and mechanical adjustmentare used such that digital pan and tilt adjustment makes minor and rapidadjustments while mechanical pan and tilt adjustment allows for largepan and tilt movements.

In another embodiment, the camera assembly is inserted into the abdomenas a separate unit from the rest of the device. This separate cameraassembly may removably couple with the device once inside of theabdominal cavity, or it may serve as a stand-alone unit.

In some embodiments, the cameras have wide-angle lenses allowing for awide visualization of the operating field. In other embodiments, thecameras have aspherical lenses allowing for a wide vertical view with anarrow horizontal view. Distortion is removed with digital adjustment.This wide vertical view allows for a tilt motion to be provided solelyusing digital technique. In yet another embodiment, the camera body 125comprises a plurality of camera devices further increasing the field ofview. Camera views are digitally interlaced to form one large image witha panoramic view. Standard digital technique known in the field is usedto interlace images. In another embodiment, the camera body additionallycomprises other sensors sensing any of pressure, capacitance,temperature, infrared, ultraviolet, or any other sensor device.

In one embodiment, the camera body 125 further comprises an array ofLEDs positioned between one camera 128 and the second camera 129. TheseLEDs serve to illuminate the operating field. These LEDs are powered viawires fed to the outside of the body. Heat from the LEDs is dissipatedwithin the camera body. In some embodiments, a small amount of sterilesaline or other biocompatible fluid may flow through the camera body tocool the camera body. Other embodiments further comprise a temperaturesensor to ensure the camera body remains within a safe temperaturerange. In another embodiment LEDs are placed within other bodies of thedevice providing for different angles of lighting as well as largerheat-sink bodies.

It is thought that the abdomen may also be illuminated via fiber opticsor another lighting source. Fiber optics may be fed into the body withactuation cables, or through another incision. In one embodiment,optical fibers are threaded into the abdomen through very small tubessuch as 21-gauge angiocatheters. Fibers could mate with the deviceinside of the abdomen, or could serve to provide illumination withoutmating with the device. Such an illumination system would provide forincreased lighting with reduced heat, but at the cost of increasedcomplexity of the overall system.

The camera body is inserted with its field of view perpendicular to thedirection of insertion through the trocar. This allows placement ofcameras on or in the camera body 125 (FIG. 6A) such that theinter-camera distance 130 exceeds the size of the incision through whichthe device is inserted. With increased inter-camera distance, the camerasystem has increased ability to visualize parallax and allow a user toperceive depth. The inter-camera distance is chosen to maintain anatural and human-like system such that

$\frac{{length}\mspace{14mu}{of}\mspace{14mu}{human}\mspace{14mu}{arm}}{{human}\mspace{14mu}{interpupillary}\mspace{14mu}{distance}}$${approximately}\mspace{14mu}{equals}\frac{{length}\mspace{14mu}{of}\mspace{14mu}{robotic}\mspace{14mu}{arm}}{{inter}\text{-}{camera}\mspace{14mu}{distance}}$Human Interaction with Device

A natural human-machine interface (HMI) was designed to best utilize thehuman-like robotic device and natural visualization system. Oneembodiment allows the surgeon to control the device with movement of hisor her own arms. The surgeon wears a series of elastic bands; each bandfastens a sensor to the surgeon's arms. In the one embodiment, thissensor is an MPU-6050 sensor. The MPU-6050 includes a MEMS gyroscope,accelerometer, and digital motion processor to compute the orientationof the sensor.

Referring to FIG. 15-A, in one embodiment, the surgeon wears eightelastic bands 226 and 227. These bands fasten eight MPU-6050 sensors tothe surgeon's arms as shown in FIG. 15. One band is placed on each ofthe right and left index finger 224, hand dorsum 223, distal dorsalforearm 222, and distal dorsal upper arm 221. The enclosure containingeach upper arm sensor additionally contains a microcontroller, battery,and Bluetooth module. Data from distal sensors is collected using I2Cprotocol along wires 225 and transmitted over Bluetooth to a centralcomputer.

With data from the eight MPU-6050 sensors, the central computer is ableto compute the position and orientation of each portion of the surgeon'sarm. Future solutions include tracking of the surgeon's torso or anyother body part. Additionally, an alternate embodiment includes theaddition of a MEMS magnetometer with each accelerometer, gyroscope, andmotion processor unit. MEMS chips such as the MPU-9250 offer all of theabove in a single package. The addition of a magnetometer is standardpractice in the field as magnetic heading allows for reduction in sensordrift about the vertical axis. Alternate embodiments also includesensors placed in surgical material such as gloves, surgical scrubs, ora surgical gown. These sensors may be reusable or disposable.

Yet another embodiment includes the addition of sensors to track theposition of the surgeon's arms and body. Such sensors, similar to thesensors in the Xbox Kinect® allow tracking of the absolute position ofthe surgeon's arms and tracking of the arms positions relative to eachother. In some embodiments, these additional sensors are worn on thesurgeon's body. In other embodiments, sensors are positioned at fixedlocations in the room.

With the ability the track the surgeon's arms, a control loop within acentral computer drives the servomotors controlling the human-likerobotic arms of the device. This can be seen in the block diagram ofFIG. 16. Arms are controlled to follow the scaled-down movement of thesurgeon's arms. The robotic elbow follows position and orientation ofthe human elbow. The robotic wrist follows position and orientation ofthe human wrist. Surgical end-effectors follow the movement of thesurgeon's index finger as the surgeon pinches their index finger andthumb together.

While the device's arms follow movement of the surgeon's arms, thedevice's shoulders are fixed in position. In one embodiment, theposition and orientation of the surgeon's torso is subtracted from theposition and orientation of the surgeon's arms. This subtraction allowsthe surgeon the move his or her torso without the arms moving. Alternateembodiments include a chair with pads to encourage the surgeon to keephis or her shoulders in fixed in space. By preventing the surgeon frommoving his or her shoulders, the surgeon avoids making movements thatthe device is unable to replicate, thus increasing the natural feel ofthe device.

The surgeon wears a virtual-reality head-mounted display 220 (FIG. 15)in order to best visualize the device. Appropriate head-mounted displayssuch as the Oculus Rift provide the user with a head-mounted display,lenses to allow focused view of the display, and a sensor system toprovide position and orientation tracking of the display. Position andorientation sensor systems may include accelerometers, gyroscopes,magnetometers, motion processors, infrared tracking, computer vision,any other method of tracking at least one of position and orientation,or any combination thereof. With many displays emerging on the market,it is important to choose the best for our system. Display featurestypically resulting in improved device function for our system includeincreased flied of view, decreased latency, decreased persistence,increased resolution, decreased weight, increased comfort, and improveddisplay position and orientation tracking.

With a head-mounted display, a computer processes video collected fromthe device's visualization system as seen in the block diagram of FIG.16. In one embodiment, video from both first and second camera 128 and129 (FIG. 6A) is collected and processed as described later on in thissection. Processed video from one camera 128 is displayed to thesurgeon's right eye. Similarly, processed video from one camera 129 isdisplayed to the surgeon's left eye. The combination of left and righteye view from separate cameras spaced apart in the abdominal cavityprovides the surgeon with stereoscopic view.

In order to maintain a full virtual reality experience, a sensor systemtracks the position and orientation of the surgeon's head mounteddisplay. This sensor system relays data to a central computer in realtime. The central computer adjusts the pan and tilt of the device'scamera system as quickly as possible to follow the movement of theuser's head. As it is difficult to adjust the pan and tilt of the camerafast enough such that the surgeon cannot perceive a delay, softwareadjusts the camera views slightly to compensate for any differencebetween the camera position and the surgeon's head position.

While some embodiments provide only visual feedback to the surgeon,alternative embodiments provide numerous additional feedback systems. Inone embodiment, the surgeon holds a device to provide haptic feedback.Such a device could be as simple as a small servomotor connected to twomembers. When the surgeon squeezes between the members, the servomotorresists the movement. With a servomotor providing haptic feedback aswell as with position and force sensing in the robotic grasper, standardforce control algorithms may be used to enable the surgeon to “feel” theforce exerted by the grasper.

In an alternate embodiment, the surgeon is provided with anexoskeleton-like device to wear on each of his or her arms. Such adevice would contain a servo for each actuator of the robotic arms andwould allow the surgeon to experience haptic feedback for each roboticactuator. In yet another embodiment the surgeon interacts with thedevice using standard haptic interaction devices known on the markettoday.

In one embodiment, motion from the surgeon's arms is translated intomotion of the device's arms with only direct scaling. However, otherembodiments may include adjustable scaling of the motion. In oneembodiment, motion is further scaled down such that a movement of thesurgeons elbow by 10 degrees results in a similar movement of thedevice's elbow by 5 degrees. This scaling allows for increased dexterityin exchange for decreased natural feel of the device. Another embodimentincludes adjustable scaling wherein the scale factor is linked to thespeed of movement. For example, if the surgeon's elbow moves 10 degreesat 10 degrees per second, the device's elbow moves 3 degrees. If thesurgeon's elbow moves 10 degrees at 50 degrees per second, the device'selbow moves 15 degrees.

The block diagram of FIG. 16 provides an overall view displaying how thedevice as a whole collects and uses information. Sensors 229 track thesurgeons body motion and relay this information to the central computer.The central computer contains control loops 230 and video rendering andaugmentation software and hardware 231. Information about the surgeon'sarm and body locations are used to calculate intended robotic actuatorpositions. Control loops continue to calculate power outputs toservomotors 232 using desired robotic actuator positions combined withprocess values from servomotor encoders, servomotor torque, deviceencoders, and any other relevant systems. These control loops may usestandard tuned proportional integral derivative “PID” control todetermine power output to servomotors. Alternatively, custom controlloops may be used. Servomotors connect with the device 233 inside of thepatient as described later in this application.

The device inside of the patient collects video signals from camerasystems and transmit these signals to the central computer's videorendering and augmentation system 231. This system combines informationabout the cameras' positions and orientations, the video signals, andthe surgeon's head position and orientation. With this information, thevideo rendering and augmentation system creates a video signal andtransmits this signal to the surgeon's virtual reality display 228. Itshould be noted that this block diagram generally describes the device,alternative embodiments have additional sensors and elements as well asadditional connections between block diagram components to allow formore complex processing and use of data within the system.

Reality Augmentation

In order to further enhance the surgeon's operating capability, realitymay be augmented to provide increased information to the surgeon. Thisaugmentation of reality serves to further the surgeon's ability tooperate by adding to the virtual reality experience. For example, in oneembodiment, the device's cameras have a zoom function. For the surgeonto use this zoom function during an operation would be unnatural, as thesurgeon's own eyes are unable to zoom on command. However, usinganimation, a surgeon may choose a magnifying glass or loupe, and bringthe glass in front of his or her virtual eyes during use. This augmentedreality allows the surgeon to feel as if he or she caused the increasein zoom, thus maintaining the natural virtual reality connection betweensurgeon and device.

In another embodiment, the surgeon is able to place augmented realityelements within the patient's abdomen. For example, to view a patient'sradiographic scans, a surgeon may choose to place a virtual computermonitor within the patient's abdominal cavity (likely in an area outsidethe field of operation). This virtual reality monitor allows the surgeonto flip through images without leaving the virtual reality of theoperation.

In another embodiment, a computer tracks the position of the roboticarms within the surgeon's field of view. If the surgeon exerts excessiveforce, the computer augments the color of the robotic arms to appear redwithin the surgeons view. Similarly, the robotic arms or surgicalend-effector may be set to change color when the surgeon enables cauteryon a cautery instrument.

Center Connection System

FIG. 7A and FIG. 7B show the center connection system. The centerconnection system serves multiple purposes, including support of thearms, support of the camera assembly 102, and routing of cables andpower systems. The center connection system comprises a main center body131 connected to the conduit 100. This connection is fashioned via anystandard attachment method known to those in the field such as a spline,press-fit, glue, weld, or any other existing attachment means. FIG. 7Bshows cable tracks 137 for connection wires used in the embodiment withfour arms as discussed later. Front cover 133 retains cables within thetracks.

Cables entering the patient's body within the conduit are routed toappropriate actuators via a system of pulleys 132 as shown in theexploded view of FIG. 7C. These pulleys captivate cables withinv-grooves 135 such that cables remain in place even when slacked. Axles136 for the pulleys are either machined in place or fit into pre-drilledholes. A rear cover 134 is bolted into place with bolts threading intotaped holes 138.

While the center connection system as used in one embodiment allows forthe insertion of the entire system through one trocar, alternativeembodiments allow for multiple single-arm units to be inserted throughseparate trocars. For example, three trocars may be used to introducetwo human-like robotic manipulators and a virtual reality camera. Thisconfiguration would accomplish the same virtual reality surgery withoutthe need for the center connection system. However, it would increasethe number of incisions required in the patient and reduce themaneuverability of the device once inserted.

Hinge Actuator Design

The first actuator 110 (FIG. 2A), the third actuator 112, the fifthactuator 114, and the seventh actuator 116 are hinge actuators. In oneembodiment, hinge actuators are cable-driven in order to provideappropriate torque and speed in the smallest possible space. This smallactuator design allows for insertion through a very small incision. FIG.8A and FIG. 8B show a perspective view of a hinge actuator. This hingeactuator includes two female components 139 with a plurality of boltholes 140 for attachment of the female components to another actuator orother device element. This attachment forms a proximal connectioncomponent. Alternative designs include attachment with other knownmeans. These means may include fabrication of the female hingecomponents and proximally attached components as a single body. Thefemale components are spaced apart leaving an area for the malecomponent of the actuator. Each female component includes a plurality ofstring guide holes 142 for passing of strings, cables wires, and othersystems through the actuator from proximally connected systems to distalactuators. In alternative designs, string guide holes are replaced withslots or a single hole to allow systems to similar pass through.Additionally, each female hinge component includes a female bearingsurface 141. In one embodiment, the female bearing surface comprises asmooth machined surface. However alternative designs include ballbearings, needle bearings, fluid bearings, or any other bearing type.

FIG. 8A and FIG. 8B further show a main hinge body 143. This main hingebody includes a distal connection component comprising a plurality ofbosses 144 on its distal end used for connection to further actuators orsystems. These bosses include a plurality of tapped holes 145 to allowattachment of distal systems to the actuator bosses. While oneembodiment includes attachment with bolts, any means of attachment couldbe appropriate including fabrication of distally attached components asa single body with the main hinge body. The main hinge body includes aplurality of string guide holes 153 for passing of strings, cableswires, and other systems through the actuator from proximally connectedsystems to distal actuators. These string guide holes function in thesame manor to the string guide holes 142 of the female components. Themain hinge body further comprises a pulley or capstan 146. This capstanis the means by which the hinge joint is actuated. Two actuation cablesare fed to the pulley along paths 147 from within conduits 149 in theproximally attached body. The cables continue to wrap around the pulleyand terminate at a cable termination site 148.

In one embodiment, cables terminate by means of clamping the cablebetween a rigid surface and a setscrew placed in tapped hole 150.Alternatively, a cable may terminate using any appropriate means knownin the field. For example a polymer fiber cable may terminate with aknot tied in the cable, or a metal fiber cable may be terminated bymeans of crimp connections. In one embodiment, each of the twodirections of actuation are provided with independent cables threadedaround the pulley in opposite directions. An alternate design comprisesa single cable entering via a first actuation cable conduit hole 149,wrapping once or a plurality of times around the pulley, and exiting viathe second cable conduit hole. While this design eliminates the need forcable termination within the actuator, it allows for slippage of thecable against the pulley. Either embodiment could be appropriatedepending on the particular application. In another embodiment a singlecable actuates the hinge actuator in one direction, while a spring orother energy storage device provides actuation in the oppositedirection.

FIG. 8A and FIG. 8B further show a contoured profile surface 152fabricated into both sides of the main hinge body 143. This contouredprofile allows actuator cables, wires, and any other systems to passthrough the hinge joint from proximal actuators and devices to distalactuators and devices. FIG. 9A and FIG. 9B show section viewsdemonstrating the function of the contour pathway. The pathway iscontoured such that the neutral axis of pass-through cables and systems158 (FIG. 9A and FIG. 9B) remains at approximately the same length atall times. The length is relatively unchanged while the main hinge body143 rotates relative to the female hinge components. This contouredshape was produced using CAD analysis of a cable passing through theactuator. The contour shape was adjusted until the cable remained withinabout 1% of its starting length at all positions of actuation, in oneembodiment. In present embodiments, the contoured surface is fashionedfrom aluminum. Polymer fiber cables pass along the surface. Alternativedesigns include any of surfaces fashioned from low friction materials(or coated with low friction materials), cables coated with low-frictionmaterials, or surfaces fashioned from wear resistant materials.Additionally, surfaces may further comprise a plurality of captivatedrollers, balls, or pulleys to further reduce friction between thesurface and cable.

With the addition of the contoured surface 152, hinge actuators mayactuate through a wide angular range without consequentially actuatingdistal systems. The contoured surface allows for dozens of cables topass through the hinge actuator without coupled motion. This allows formany cable driven actuators to be attached distally to the hingeactuator without any significant coupled motion between actuators.Alternative actuator designs used in surgery today at most allow a fewcables to pass with moderately coupled motion. This contoured surfaceallows for almost entirely decoupled motion with many cables, thuspermitting a seven degree of freedom robotic arm to fit through a 12 mmtrocar.

In some embodiments, the change of length of the actuator cable withinthe actuator is less than about 10% as the actuator moves through arange of motion of 110 degrees. In other embodiments, the change oflength is less than about 9%. In still further embodiments, the changeof length is less than about 8%. In still further embodiments, thechange of length is less than about 7%. In still further embodiments,the change of length is less than about 6%. In still furtherembodiments, the change of length is less than about 5%. In stillfurther embodiments, the change of length is less than about 4%. Instill further embodiments, the change of length is less than about 3%.In still further embodiments, the change of length is less than about2%. In still further embodiments, the change of length is less thanabout 1%.

Each hinge actuator further comprises two male hinge bodies 154 as shownin FIG. 8B. Each male hinge body has a contorted profile surface 152with profile identical to the contorted profile surface of the mainhinge body 143. This profile surface provides function identical to theprofile surface of the main hinge body. Additionally, each male hingebody comprises a plurality of bolt holes 157 for attachment of the malehinge body to the main hinge body. Bolts are threaded into tapped holes151 on the main hinge body semi-permanently fixing the male hinge bodyto the main hinge body. The male hinge body further comprises a maleboss 155 acting as a hinge-pin for the hinge joint. A bearing surface156 along the male boss acts in conjunction with the bearing surface 141on the female hinge component 139. These surfaces provide a bearing forthe hinge joint. Bearings work in conjunction to form a bearing system.As with the female bearing surface, the male bearing surface may be acylindrical machined surface, or alternatively may include races and anytype of bearing such as ball, needle, fluid, etc.

While one embodiment fastens the male hinge bodies 154 to the main hingebody 143 with a plurality of bolts, any alternative fastening methodcould be acceptable. One alternate design fastens the bodies by means ofadhesive. In another design, the bodies are fabricated as a single bodythus eliminating the need for a means of fastening. Fabrication as onepart would increase complexity of fabrication operations while reducingcomplexity of assembly.

In one embodiment, all hinge actuators are designed to be identical,however alternative embodiments include different hinge actuators tomeet specific needs of one, third, fifth, and seventh actuators. Forexample, a first actuator might require a range of motion from negative50 degrees to positive 50 degrees. Thus, the actuator as presentlydesigned would function ideally. However, a fifth actuator (representingthe elbow joint) may need a range of motion from about 0 degrees to 160degrees. Thus, a modified actuator with an appropriate range of motionwould be ideal for this joint.

Additionally, alternative embodiments could include hinge-type actuatorsdiffering significantly from the actuators of one embodiment. Actuatorscould include motors, gearboxes, pistons, or any other means of jointactuation.

In another embodiment, the actuator further comprises an encoder formeasurement of position at the joint. Measurement is obtained with aHall-effect encoder similar to the AS5055 by ams AG. Such an encodereasily fits within the hinge joint, and provides for real-timemeasurement of position with 12-bit resolution. Alternatively any meansof measuring position may be used including optical and resistiveencoders. Data is communicated to systems outside of the abdominalcavity via wires. In some embodiments, data is communicated wirelessly.In other embodiments, data is communicated using conductive actuatorcables as wires. Actuator cables or individual strands thereof arecoated with an electrical insulator allowing the cable to transmit atleast one of electrical power and data in addition to transmission ofmechanical power.

Alternative embodiments further comprise at least one strain gauge fixedon a member experiencing strain during loading of the hinge joint. Sucha strain gauge allows for measurement of force experienced by theactuator.

Rotary Actuator Design

The second, fourth, and sixth actuators 111, 113, 115 (FIG. 2A) of oneembodiment are rotary-type actuators. FIG. 10A, FIG. 10B, and FIG. 10Cshow views of the rotary actuator of one embodiment. As with the hingeactuators used in one embodiment, rotary actuators are cable driven.Actuators are designed to provide for maximum torque and speed withminimum size.

Rotary actuators comprise a rotary female body 160. The rotary femalebody has a proximal connection component comprising a plurality ofproximal connection bosses 179 each with a plurality of bolt holes 172.Bolts fasten the proximal connection bosses to a proximal actuator orstructure. While one embodiment uses bolts as a means of fastening,various alternative means of fastening are acceptable. In onealternative design, the rotary female body and the proximal structure towhich it is fastened are fabricated as a single body.

A rotary male body 159 is inserted within the rotary female body 160.The rotary male body has a distal connection component comprising distalconnection bosses 180 (FIG. 10A and FIG. 10B) each with a plurality oftaped holes 168 (FIG. 10C). Bolts threaded into the taped holes serve toconnect distal systems with the rotary male body. Again, any alternativeattachment means may be appropriate.

The rotary male body is constrained relative to the rotary female bodyby means of a bearing system comprising two ball bearings, a small ballbearing 161 and a large ball bearing 162 (FIG. 10A and FIG. 10B). Theseball bearings act to support both axial loads and radial loads by meansof V-shaped bearing races 173 and 174 best seen on the section view ofFIG. 10D. With V-shaped races, the bearings are able to take substantialaxial and radial loads, however the bearings are no longer perfectrolling element bearings. A small amount of slip is experienced as thebearing balls 181 (FIG. 10C) roll along the bearing race, however thisslip is minimized with very small balls. In one embodiment, the bearingballs are approximately 1 mm in diameter. Wear is of minimal concern duethe low RPM of the device.

To minimize the size of the device, ball bearings are built into thedevice. The large ball bearing 162 has two bearing races 173. One raceis formed into the distal surface of the rotary female body while theother race is formed into a large bearing race ring 163. While oneembodiment comprises a bearing race formed into a separate ring in orderto decrease manufacturing cost, alternative embodiments include abearing race formed directly into the rotary male body. This alternativedesign increases complexity of manufacturing while decreasing devicesize.

The small ball bearing 161 is placed at the proximal end of theactuator. This small bearing serves to constrain the rotary male body159 within the rotary female body 160 on the proximal side of the rotaryactuator. This small ball bearing is smaller than the large ball bearing162 so as to fit between the proximal connection bosses 179. The smallball bearing comprises V-groove races formed into the small ball bearingrace ring 164 and into the splined ball bearing race ring 165. Thesplined ball bearing race has a spline on its inner surface that mateswith a spline formed onto the surface of the proximal end of the rotarymale body. Finally, a nut 166 is threaded onto the proximal surface ofthe rotary male body. This nut compresses both the small ball bearingand the large ball bearing and serves to sandwich the rotary female bodybetween the two bearings with a pre-strain force applied by rotating thenut. This pre-strain force constrains the rotary male body to the rotaryfemale body in all degrees of freedom except for the intended axis ofactuation.

Additionally, the splined surface between the rotary male body and thesplined ball bearing race ring 165 prevents the splined ball bearingrace ring from rotating relative to the rotary male body during use.This removes any torque on the nut that might loosen the nut over time.In an alternative embodiment, the nut is held in place with threadlocker, glue, a cotter pin, or any other means. In yet anotheralternative embodiment, the nut is replaced with an E-clip or othermeans of attachment, and the rotary male body may be strained duringattachment to provide for pre-strain of the bearings. In yet anotherembodiment, one or both of the small bearing race ring and the splinedbearing race ring are removed, and the bearing races are formed one ormore of the nut and the rotary female body. This change reduces theoverall length of the device and the number of device components.

It should be noted that while both the small ball bearing and the largeball bearing in the rotary cable actuator are V-raced ball bearings, anytype of bearing that can support the appropriate loads could be used.For example, an alternative embodiment uses four ball bearings, two eachfor thrust and axial loads. Another alternative embodiment uses plainbearings for axial loads and rolling element bearings for thrust loads.Yet another embodiment uses only plain bearings for axial and thrustloads. While plain bearings have increased friction, their reduced sizeand complexity is beneficial in some applications. Another embodimentincludes tapered needle bearings, such bearings are theoretically idealfor the type of loads experienced within the rotary cable actuator, yetare more difficult and expensive to manufacture.

With the rotary male body 159 fixed within the rotary female body 160,actuation cables are fed along a path from the proximally attachedactuator. These cables are fed into the space between the rotary malebody and the rotary female body. FIG. 10D shows a section view of therotary cable actuator. In this section, it is easy to view the cableseparation ridges 175 formed into the rotary female body 160. Theseparation ridges serve to form nearly fully enclosed pockets 176 foreach actuator cable to wrap around the rotary male body 159.

Each of the actuation cables follows a contoured pathway 169 and 177into the actuator (FIG. 10A and FIG. 10B). These pathways allow theactuator cables to enter the enclosed pockets 176 (FIG. 10D). Thepathways are formed into the rotary female body 160 and are contouredsuch that the cable follows a smooth path into the enclosed pockets andaround the rotary male body. It is important to note that each of thetwo actuator cables wrap around the rotary male body in oppositedirections. Thus, the rotary male body forms two pulleys, one foractuation in each rotational direction. The actuation cables each wraparound the rotary male body a plurality of times allowing for numerousrevolutions of actuation. After wrapping around the rotary male body,the actuation cables enter the rotary male body via cable terminationholes 171 and are rigidly attached to the rotary male body with means ofset screws placed in tapped holes. Holes 170 in the rotary female bodyallow a wrench to access the set screws.

Alternative embodiments of the rotary actuator include attachment of theterminated actuation cable with any alternative means appropriate. Forexample cables may be terminated with knots tied in the cable. Otheralternative termination means include glue, crimp connections, clamps,capstans, welding, and any other appropriate means. Alternatively theactuation cables may not be terminated at all. A single actuation cablemay be inserted from one side of the actuator, make a plurality of wrapsabout the rotary male body, and exit the other side of the actuator.Such a rotational actuator would allow for continuous rotation, yetwould also allow for slipping of the cable along the rotary male body.

The center of the rotary male body is fashioned to be hollow. This formsa hole 178 through the actuator. This hole is best visualized in thesection of FIG. 10D, but is also visible in FIG. 10C. The hole iscritical to the function of the device as it allows cables and othersystems to pass through the rotary actuator from proximal actuators andsystems to distal actuators and systems. Chamfer 167 reduces wear oncables by removing a sharp edge as cables pass through the hole. Byproviding a pathway straight through the center of the rotary actuator,cables pass through with almost no change in length as the rotaryactuator changes position. This is very important as it nearly fullydecouples the distal actuators from the motion of the rotary actuator.It should be noted that some amount of coupled motion occurs due totwisting of the cables passing through the hole 178, however thistwisting provides a negligible effect for actuations within one fullrotation (360 degrees) and very little effect for actuations within twofull rotations. Of course, with more than one cable passing through thehole, the rotary actuator is unable to perform infinite rotations due totwisting of the bundle of cables.

The inclusion of hole 178 within the cable driven rotary actuatorenables the many degree of freedom device to fit through a 12 mm trocar.By providing a miniature cable-driven actuator with a means to allowcables to pass through the actuator with nearly decoupled motion,multiple rotary actuators may be used in series within a robotic arm.Cables from distally attached rotary actuators as well as any distallyattached systems may simply pass through hole 178 thus allowing for adaisy chain of actuators with sufficiently decoupled motion.

In some embodiments, the change of length of the actuator cable withinthe actuator is less than about 20% as the actuator moves through arange of motion of 360 degrees. In other embodiments, the change oflength is less than about 19%. In still further embodiments, the changeof length is less than about 18%. In still further embodiments, thechange of length is less than about 17%. In still further embodiments,the change of length is less than about 16%. In still furtherembodiments, the change of length is less than about 15%. In stillfurther embodiments, the change of length is less than about 14%. Instill further embodiments, the change of length is less than about 13%.In still further embodiments, the change of length is less than about12%. In still further embodiments, the change of length is less thanabout 11%. In still further embodiments, the change of length is lessthan about 10%. In other embodiments, the change of length is less thanabout 9%. In still further embodiments, the change of length is lessthan about 8%. In still further embodiments, the change of length isless than about 7%. In still further embodiments, the change of lengthis less than about 6%. In still further embodiments, the change oflength is less than about 5%. In still further embodiments, the changeof length is less than about 4%. In still further embodiments, thechange of length is less than about 3%. In still further embodiments,the change of length is less than about 2%. In still furtherembodiments, the change of length is less than about 1%.

Grasper Design

In order to allow for maximum utility of the device, the end-effectorsof one embodiment are designed to accomplish multiple tasks. Duringnon-robotic minimally invasive surgery, a surgeon can simply remove aninstrument and insert another instrument. However, with our surgicaldevice as described in one embodiment, removal of the device to changeend-effector might be impractical. For this reason, a general-purposeend-effector is incorporated into one embodiment.

During a typical surgical procedure, a surgeon must grasp both softtissue and hard instruments. For example, a surgeon might want to movetwo soft sections of intestine together, and then sew with a small andhard needle. Grasping intestine requires a grasper capable of delicatemanipulation while grasping a needle requires high force. In order tomaximize the capability of the single end-effector, the grasper of oneembodiment was designed to grasp with variable force.

FIG. 11A shows a perspective view of the grasper in one embodiment. FIG.11A additionally shows the attached distal hinge joint, the seventhactuator 116 (FIG. 2A) with female hinge components 139 identical tothose of other hinge joints in the device. The grasper is comprised oftwo jaws 187 and 188 designed to provide sufficient length forgeneral-purpose tissue manipulation. In one embodiment the grasper jawshave ridged teeth 189 to allow for additional grasping traction.Alternatively, the graspers may be designed with any appropriate surfaceknown in the field.

The jaws are held in place between a male grasper body 185 and a femalegrasper body 186 and flexural clamping bodies 182 and 183. Together, themale grasper body, female grasper body, and flexural clamping bodiesform a main grasper body. Alternative embodiments include a main grasperbody comprising a single body, fewer bodies, or more bodies. Thesebodies support the components within the grasper and serve to form thedistal section of the hinge joint comprising the seventh actuator 116.In order to provide for the minimum possible distance between thegrasper jaws and the seventh actuator, the distal portion of the hingejoint is formed into the male grasper body 185 and the female grasperbody 186. In alternative embodiments, a separate hinge joint is used andthe grasper bodies instead include an attachment site to fasten thegrasper bodies' proximal ends to the hinge joint's distal end.

In one embodiment, male grasper body 185 is fastened to the femalegrasper body 186 via mating features 198 as well as a plurality of boltsthreaded into tapped holes. In alternate embodiments the bodies may befastened together via means such as adhesive, or parts may bemanufactured as one body. In addition to housing the grasper, theassembled male grasper body and female grasper body together with malehinge bodies 184 form the distal portion of a hinge. This hinge jointfunctions in the same manner as other hinge joints described in thesystem.

FIG. 11B shows an exploded view of the grasper. At the center of thegrasper are the grasper jaws 187 and 188. These jaws are coupled to themale grasper body 185 and the female grasper body 186 via pins 199, 200,201, and 202. These pins slip-fit into holes 197 in the grasper jaws. Inone embodiment, the pins do not extend to the center of the grasperjaws, thus maintaining an open cutaway slot 207 within the center of thegrasper jaws.

Pins 203 are formed into the end of each grasper jaw 187 and 188 (FIG.11B). These pins slip-fit into holes 204 of linkage members 190 and 191forming hinge connections. Each linkage member connects a grasper jawpin 203 with one of the pulley pins 205 formed into the body of eachpulley 192 and 193. Additionally, pulley pins 210 move within slotsformed into male grasper body 185 and the female grasper body 186. Theslot 206 formed into the male grasper body is clearly seen in FIG. 11A.An identical slot (not pictured) is formed into the female grasper body.

The linkage mechanisms formed by the aforementioned grasper componentsform linkages such that each pulley 192 and 193 is constrained tomovement in a straight line in the distal< >proximal directions. Anactuation cable is fed from the main grasper body around each pulley andto a termination site located proximally within the main grasper body.When each actuator cable is tensioned, it pulls its respective pulley inthe proximal direction. The pulley for each linkage provides for a 2-1mechanical advantage via a block and tackle mechanism. Alternativeembodiments include alternative pulley arrangements or no pulley at allfor 1-1 mechanical advantage. In one embodiment, the cables are clampedin a threaded hole by a setscrew. In other embodiments the cables may beterminated by any means known in the field such as knots, flexuralclamps or adhesives.

In one embodiment, there are two pulleys, a distal pulley 192 and aproximal pulley 193. As the grasper jaws move, the pulleys move inopposite directions. The relevant linkages can be seen in thecross-sections of FIG. 11C and FIG. 11D. As the linkage diagrams in FIG.12A and FIG. 12B show, pulleys separate as the grasper tips separate.Pulleys move together as the grasper tips move together. When a firstcable is tensioned as indicated by arrow 211, it pulls the proximalpulley in the proximal direction. Force is transmitted through theproximal linkages 214 which in turn causes the grasper jaw tips 208 toseparate in the direction of arrows 209 as depicted in FIG. 12A. When asecond cable is tensioned as indicated by arrow 212, it pulls the distalpulley in the proximal direction. Force is transmitted through thedistal linkages 213 which in turn causes grasper jaw tips 208 to movetogether in the direction of arrows 210 as depicted in FIG. 12B. Thegrasper mechanism in its current embodiment has a non-linearrelationship between movement of the actuator cables and movement of thegrasper jaw tips. This non-linear movement provides the highestmechanical advantage when the jaws are closed. This has the benefit ofenabling the grasper to handle the high loads that it sees during thehandling of small tools such as needles while also providing for lessforce while grasping large objects such as tissue.

In alternative embodiments, a single linkage and pulley are used to movethe grasper tips in both directions. One cable pulls from the proximaldirection and one cable from the distal direction. While oneembodiment's dual-linkage allows for both cables to pull from theproximal direction thus simplifying cable routing, it has increasedcomplexity and more parts. In another embodiment, a spring or otherenergy storage device provides actuation in one direction while a singleactuation cable provides actuation in the opposite direction.

In one embodiment, jaws 187 and 188 (FIG. 11B) as well as all connectedlinkages and pulleys are inserted into the assembled male grasper bodyand female grasper body. As previously described, these parts are fixedin place by the pins formed into flexural clamping bodies 182 and 183.Pins 199 and 200 retaining one grasper jaw 187 are rigidly attached tothe flexural clamping bodies. However pins 201 and 202 retaining thesecond grasper 188 are not rigidly fixed to the flexural clampingbodies. Flexures 194 and 195 link the pins 201 and 202 to theirrespective flexural clamping bodies.

When force is exerted on the grasper tips, it provides a force acting todisplace the position of pins 199, 200, 201, and 202. While pins 199 and200 remain rigidly fixed, pins 201 and 202 displace slightly under theload as the flexures are elastically deformed. The exact displacement ofthe flexures can be measured by means of strain gauges fixed to theflexures and positioned within the strain gauge pockets 196 (FIG. 11A),and another similar pocket on the other side of the device, not visiblein the figures. Standard technique may be used to acquire informationand calculate flexure strain and grasper forces

The size of the flexure is chosen such that the strain gauges are ableto measure forces appropriate for the application. In the event that thegrasper force exceeds the elastic deformation range of the flexures, theflexures contact either side of the strain gauge pockets and arehard-stopped.

Using the dimensions and mechanical properties of the flexure, forceacting at the end of grasper jaws is calculated continuously. Force canbe fed-back to the surgeon via various means such as haptic feedback orvia visual cues on the control console. Additionally, force controlalgorithms may provide closed loop control of grasper force. Softwarelimits are enabled to prevent damage to tissue by exertion ofunacceptable forces. These limits may be changed during a procedure byeither manual or automated means.

Alternative embodiments include strain gauges placed elsewhere in thesystem such as on connection linkages or the graspers themselves. Trueload cells may be designed into elements of the grasper thus providingincreased accuracy and repeatability of measurements.

Arm Length Adjustment

In one embodiment, robotic arms 103 and 104 (FIG. 1A) are configuredwith extension segments 108 to increase arm length. This increased armlength allows for improved reach within the abdominal cavity. Extensionsare used to allow for adjustment of arm length without modification toactuator design. For example, an embodiment of the device for pediatricuse might require smaller arms for maneuverability within a smallerabdominal cavity. Such a device could be fashioned by removing the armextensions. Alternate embodiments with very significant length changerequire modification to camera position and shoulder width in order tomaintain human-like proportions.

FIG. 13 shows a view of the extension segments used in one embodiment.Such extension segments are formed from a single piece of metal withproximal and distal ends to allow connection to the existing proximalactuator at a proximal connection site 215 and distal actuator at adistal connection site 216. Additionally, a center channel 217 formed inthe device allows cables to pass through the extension from the proximalactuator to the distal actuator.

Complex surgeries often require more than one surgeon for operation.With this in mind, we designed a four-arm embodiment of the device foruse by two surgeons. FIG. 14A shows the device of the four-armembodiment. Such a device has four arms and two cameras, all insertedthrough a single trocar. This four-arm embodiment allows for twosurgeons to work together. In the four-arm embodiment, two devicessimilar to that of one embodiment are inserted separately into theabdominal cavity, thus allowing two surgeons to use the two roboticdevices.

FIG. 14A shows the four-arm embodiment as configured for two surgeons towork along one side of the device. In this orientation, it is as if onesurgeon is standing behind the other surgeon. Using each two-armdevice's ability to operate on either side of the device as previouslydescribed, surgeons may choose the area in which the four-arm embodimentis used. Surgeons may orient both two-arm systems to operate on eitherside of the four-arm device (with both two-arm devices facing in thesame direction). Alternatively, surgeons may orient both two-arm devicesfacing each other to operate in the area between the two devices.Surgeons may even face away from each other to operate separately oneither side of the patient.

With two devices in the abdominal cavity, optional connection linkage218 allows the devices two devices to be rigidly interconnected. Thislinkage mechanism is inserted into the abdomen in sequence with the tworobotic devices. For example, a surgeon may insert one robotic device,then insert the connection linkage, and then insert the second roboticdevice all through one trocar. The connection linkage mechanism may alsoinclude an optional center support 219. This support serves to rigidlyfix the connection linkage and thus the robotic devices to the outsideworld through a second incision in the patient's abdominal wall. Thissecond support in addition to each conduit 100 support allows forincreased device rigidity. A very thin center support may pass throughthe abdominal wall via a needle-placed catheter similar to anangiocatheter. This small catheter almost entirely eliminates injury tothe patient caused by the center support's traversal of the abdominalwall.

In one design, additional cables pass through each conduit 100 (FIG. 7B)and into each main center body 131. These cables continue to passthrough pathways 137 and to each member of the connection linkage 218(FIG. 14A) with at least one cable terminating in the center support.Said cables are given significant slack for insertion of the devices andall parts into the abdominal cavity. Once all devices parts are insertedinto the abdominal cavity, the surgeon may pull on the ends of thecables, thus pulling all members of the connection linkage together.This assembly process functions in a similar manor to assembly of acamping tent where tent pole segments remain attached by a cord.Constant mechanical force applied to these cables can serve to fastenthe connection linkages firmly together during use.

Alternatively, the surgeon may remove the connection linkage or use adevice with no provided connection linkage. Two devices as described inone embodiment could be used during a single surgery with no connectionlinkage needed. Some embodiments of a four-arm system are designed forinsertion through a single four-arm trocar with room for both conduitmembers as shown in FIG. 14B. Other embodiments are placed through twostandard two-arm trocars. Some embodiments allow software controls tointerconnect such that surgeons may use software commands to switchwhich arms they control, even choosing any one or two of the arms withinthe abdomen for control. Arms may optionally lock in place usingsoftware when not under control of a surgeon allowing a single surgeonto alternate between control of numerous devices.

In addition to the aspects of the invention claimed blow, the inventionalso includes the following aspects and embodiments. The system of theinvention may include systems to couple two two-arm robotic devicestogether. The system of the invention may also include trocars forinsertion and support of the robotic device, trocar sleeves for supportof a device conduit and trocar sleeves with a camera and lightingsystem. Sensor configurations for acquisition of surgeon arm and handmovement are also considered part of the invention. The invention isalso considered to include devices for virtual reality robotic surgerywherein robotic actuators are not permanently coupled with camerasystems and devices for virtual reality robotic surgery wherein roboticactuators are not coupled with camera systems within the abdominalcavity. The invention also encompasses virtual reality robotic devicesfor use in small parts assembly, diffusing of bombs, inspection andrepair within enclosed spaces, as well as any other use of virtualreality robotics.

Other aspects of the invention include virtual reality camera positionedrelative to robotic actuations similarly to the position of human eyesrelative to human arms and a virtual reality camera with a human-likeratio of the distance between the cameras to the size of the roboticarms. A further aspect of the invention is computer limiting of roboticmotion. Also inventive is insertion of the device using an insertionbody and an insertion body including sensors to detect contact orproximity with the patient's body. With respect to camera(s) for usewith the virtual reality surgical system inventive aspects includecamera movement within the camera body, cameras with aspherical lensescombined with computer adjustment and correction of the image, cameraswith wide angle lenses, cameras with wide angle lenses combined withcomputer adjustment and correction of the image, and cameras with theability to zoom digitally or via mechanical means. Reality augmentationdisplaying a magnifying glass or loupe to allow for natural human-likezoom and placement of augmented reality devices within the abdomen,viewing a virtual computer monitor within the abdomen, and feedback tothe surgeon via augmenting reality and the display of the robotic arm orassociated devices are also aspects of the present invention. Anotherinventive aspect of the system is the interlacing the images from manycameras to form a single image.

The cooling of systems using saline is also part of the invention.Another aspect of the invention is optical fibers providing light andinserted through a number of small incisions. Also part of the inventivesystem is scaling of the motion between the surgeon's arms and therobotic device by a constant scaling factor, scaling of the motionbetween the surgeon's arms and the robotic device by a user configurablescaling factor and scaling of the motion between the surgeon's arms andthe robotic device by a scaling factor that adjusts based on the rate ofchange in position of the surgeon's arms. Still further inventiveaspects of the system include use of a single ball bearing to providefor both axial and radial load support, actuators with a range of motionin excess of the required motion to allow the device to operate on boththe first side and the second side, and use of hall effect encoders toacquire position data from within the abdominal cavity. With respect tothe grasper inventive aspects include use of pulleys to provide formechanical advantage within the grasper, dual-linkage mechanisms withinthe grasper to allow two actuation cables to both pull from the proximalside of the grasper with one cable opening the grasper and the secondcable closing the grasper, use of flexures attached to a pivot pin ofthe grasper jaw to allow for accurate measurement of grasper force byusing a strain gauge placed along the flexure and use of a load cellsdesigned into a linkage of the grasper.

The techniques and systems disclosed herein may have certain controlcomponents implemented as a computer program product for use with acomputer system or computerized electronic device. Such implementationsmay include a series of computer instructions, or logic, fixed either ona tangible medium, such as a computer readable medium (e.g., a diskette,CD-ROM, ROM, flash memory or other memory or fixed disk) ortransmittable to a computer system or a device, via a modem or otherinterface device, such as a communications adapter connected to anetwork over a medium.

The medium may be either a tangible medium (e.g., optical or analogcommunications lines) or a medium implemented with wireless techniques(e.g., Wi-Fi, cellular, microwave, infrared or other transmissiontechniques). The series of computer instructions embodies at least partof the functionality described herein with respect to the system. Thoseskilled in the art should appreciate that such computer instructions canbe written in a number of programming languages for use with manycomputer architectures or operating systems.

Furthermore, such instructions may be stored in any tangible memorydevice, such as semiconductor, magnetic, optical or other memorydevices, and may be transmitted using any communications technology,such as optical, infrared, microwave, or other transmissiontechnologies.

It is expected that such a computer program product may be distributedas a removable medium with accompanying printed or electronicdocumentation (e.g., shrink wrapped software), preloaded with a computersystem (e.g., on system ROM or fixed disk), or distributed from a serveror electronic bulletin board over the network (e.g., the Internet orWorld Wide Web). Of course, some embodiments of the invention may beimplemented as a combination of both software (e.g., a computer programproduct) and hardware. Still other embodiments of the invention areimplemented as entirely hardware, or entirely software (e.g., a computerprogram product).

The invention claimed is:
 1. A system for use in surgery comprising: acentral body; a visualization system operably connected to the centralbody comprising: at least a first camera, and at least one of a pansystem or a tilt system; a first processor and machine readable memorycomprising instructions that when executed cause the first processor togenerate images based on information from the at least one camera; ahead-mounted display for displaying images generated by the firstprocessor; a first sensor for tracking at least one of the position inspace of the head-mounted display relative to a first reference point orthe orientation in space of the head-mounted display relative to thefirst reference point, wherein at least one of the pan system or thetilt system are configured to adjust the field of view of the camera inresponse to information from the first sensor about changes in at leastone of position or orientation in space of the head-mounted displayrelative to the first reference point; a robotic device operablyconnected to the central body, wherein the robotic device comprises: afirst rotational actuator for rotating a first portion of the roboticdevice with respect to a second portion of the robotic device; a firsthinged actuator for changing the angle between a third portion of therobotic device and a fourth portion of the robotic device operablycoupled to the first rotational actuator; a second rotational actuatorfor rotating a fifth portion of the robotic device with respect to asixth portion of the robotic device operably coupled to the first hingedactuator; a second hinged actuator for changing the angle between aseventh portion of the robotic device and an eighth portion of therobotic device operably coupled to the second rotational actuator; athird rotational actuator for rotating a ninth portion of the roboticdevice with respect to a tenth portion of the robotic device operablycoupled to the second hinged actuator; a third hinged actuator forchanging the angle between an eleventh portion of the robotic device anda twelfth portion of the robotic device operably coupled to the thirdrotational actuator; and a surgical end-effector operably coupled to thethird hinged actuator; a second sensor for tracking a series of changesin a position of a portion of an arm of a user; a second processor andmachine readable memory comprising instructions that when executed causethe second processor to determine a series of positions and orientationsof at least two of the twelve portions of the robotic device in responseto the series of changes in the position of the portion of the arm ofthe user based on the degrees of freedom of a human arm, whereby thedetermined series of positions and orientations of the portions of therobotic device replicate a human-like motion achievable by the humanarm; and at least one servomotor for adjusting at least one of positionsor orientations of at least one of the first rotational actuator, thesecond rotational actuator, the third rotational actuator, the firsthinged actuator, the second hinged actuator, the third hinged actuator,or the surgical end-effector to cause the robotic device to follow theseries of changes in position of the portion of the arm of the useraccording to the determined series of positions and orientations of theat least two of the twelve portions of the robotic device so that atleast one of the twelve portions of the robotic device mimics the motionof the arm of the user.
 2. The device of claim 1, the first processorand machine readable memory comprising instructions that when executedcause the first processor to digitally adjust the field of view of thegenerated images based on information from the first sensor.
 3. Thedevice of claim 1, the visualization system further comprising a secondcamera.
 4. The device of claim 3, the images generated by the firstprocessor comprising stereoscopic images based on information from thefirst and second cameras.
 5. The device of claim 3, wherein, in aninsertion configuration, cross-sectional dimensions of the visualizationsystem in a plane normal to an insertion axis are smaller than a centerdistance between the first camera and the second camera along theinsertion axis.
 6. The device of claim 1, comprising a plurality ofcameras.
 7. The device of claim 6, the first processor and machinereadable memory comprising instructions that when executed cause thefirst processor to generate the images based on software interlacing ofsignal information from the plurality of cameras.
 8. The device of claim1, further comprising at least one camera sensor to measure at least oneof the position and orientation of the camera.
 9. The device of claim 1,wherein the robotic device further comprises a positional actuator forchanging the position of the robotic device relative to the central bodysuch that the robotic device may be used on either a first side or asecond side of the central body.
 10. The device of claim 1, furthercomprising a second robotic device comprising: a fourth rotationalactuator for rotating a first portion of the second robotic device withrespect to a second portion of the second robotic device; a fourthhinged actuator for changing the angle between a third portion of thesecond robotic device and a fourth portion of the second robotic deviceoperably coupled to the fourth rotational actuator; a fifth rotationalactuator for rotating a fifth portion of the second robotic device withrespect to a sixth portion of the second robotic device operably coupledto the fourth hinged actuator; a fifth hinged actuator for changing theangle between a seventh portion of the second robotic device and aneighth portion of the second robotic device operably coupled to thefifth rotational actuator; a sixth rotational actuator for rotating aninth portion of the second robotic device with respect to a tenthportion of the second robotic device operably coupled to the fifthhinged actuator; a sixth hinged actuator for changing the angle betweenan eleventh portion of the second robotic device and a twelfth portionof the second robotic device operably coupled to the sixth rotationalactuator; and a second surgical end-effector operably coupled to thesixth hinged actuator.
 11. The system of claim 1, wherein at least oneof the first hinged, second hinged, and third hinged robotic actuatorscomprises: a first body comprising: a proximal connection componentcoupling the robotic actuator to proximal systems, and a first bearingsurface; a second body comprising: a distal connection componentcoupling the robotic actuator to distal systems, and a second bearingsurface forming a bearing with the first bearing surface whereby thebearing constrains the motion of the first body relative to the motionof the second body in at least one degree of freedom; at least one of apulley and a capstan operably coupled with at least one of the firstbody and the second body; an actuator cable configured to actuate atleast one of the pulley and the capstan; and at least one contouredsurface defined by the robotic actuator and forming a contoured pathwayto allow a plurality of additional cables to pass through the pathwayfrom systems coupled to the proximal connection component to systemscoupled to the distal connection component wherein a shape and aposition of the pathway is such that lengths of the additional cablesremain substantially constant for substantially an entire range ofmotion for which the robotic actuator is used.
 12. The system of claim1, wherein at least one of the first hinged, second hinged, and thirdhinged robotic actuators comprises: a first body comprising a proximalconnection component; a second body comprising a distal connectioncomponent; a bearing system constraining the motion of the first bodyrelative to the second body in all degrees of freedom except rotationabout one axis perpendicular to the distal-proximal axis of the roboticactuator; at least one of a pulley or a capstan operably coupled with atleast one of the first body and the second body; an actuator cableconfigured to actuate the at least one of the pulley and the capstan;and at least one contoured surface defined by the robotic actuator andforming a contoured pathway to allow a plurality of additional cables topass through the pathway from systems coupled to the proximal connectioncomponent to systems coupled to the distal connection component whereina shape and a position of the pathway is such that lengths of theadditional cables remain substantially constant for substantially anentire range of motion for which the robotic actuator is used.
 13. Thesystem of claim 1, wherein at least one of the first rotational, secondrotational, and third rotational robotic actuators comprises: a firstbody comprising a proximal connection component; a second bodycomprising a distal connection component; a bearing system constrainingthe motion of the first body relative to the second body in all degreesof freedom except rotation about the distal-proximal axis of the roboticactuator; at least one of a pulley or a capstan operably coupled with atleast one of the first body or the second body; an actuator cableconfigured to actuate the at least one of the pulley and the capstan;and a hole defined by the robotic actuator with an inner diameter of atleast three times the diameter of the actuator cable configured suchthat additional cables may pass through the hole from systems coupled tothe proximal connection component to systems coupled to the distalconnection component wherein a shape and a position of the hole is suchthat lengths of the additional cables remain substantially constant forsubstantially an entire range of motion for which the robotic actuatoris used.
 14. The system of claim 1, wherein the surgical end-effectorcomprises: a main grasper body; a first grasper jaw operably coupled tothe main grasper body; a second grasper jaw operably coupled to the maingrasper body; an actuation cable; and a linkage mechanism coupling atleast one of the first grasper jaw and the second grasper jaw with theactuation cable wherein the linkage mechanism provides for non-linearmovement of a distal end of at least one of the first grasper jaw or thesecond grasper jaw in response to movement of the actuation cable. 15.The surgical grasper of claim 14, further comprising a strain gaugefixed to at least one of the main grasper body, the first grasper jaw,the second grasper jaw, the actuation cable, and the linkage mechanismwherein a force value between the distal end of the first grasper jawand the distal end of the second grasper jaw is determined based oninformation from the strain gauge.
 16. The surgical grasper of claim 15,further comprising an operator interface including a haptic feedbackdevice for providing haptic feedback to a user of the operator interfacebased on the information from the strain gauge.
 17. The surgical grasperof claim 14, further comprising at least one of a spring operablycoupled with at least one of the first grasper jaw and the secondgrasper jaw.
 18. The surgical grasper of claim 14, further comprising atleast one of software and hardware control loops for controlling atleast one of the force of the grasper jaws and the position of thegrasper jaws.
 19. The surgical grasper of claim 14, further comprisingat least one of a servomotor operably coupled with the actuation cable.20. The surgical grasper of claim 14, further comprising at least one ofa position sensor for measuring the position of at least one of thefirst grasper jaw and the second grasper jaw.